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Abstract 

In  this  note  we  consider  a  large  step  modification  of  the  Mizuno-Todd-Ye  0(y/nL ) 
predictor-corrector  interior-point  algorithm  for  linear  programming.  We  demonstrate 
that  the  modified  algorithm  maintains  its  0(>/nL)-iteration  complexity,  while  ex¬ 
hibiting  superlinear  convergence  for  general  problems  and  quadratic  convergence  for 
nondegenerate  problems.  To  our  knowledge,  this  is  the  first  construction  of  a  super¬ 
linearly  convergent  algorithm  with  0(y/nL)- iteration  complexity. 
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1.  Introduction 


Consider  the  primal  linear  program  (LP): 

T 

mm  c  x 

s.t.  Ax  =  6,  x  >  0, 

and  its  dual  (LD): 

max  bT  y 

rji 

s.t.  A  y  +  s  —  c,  s  >  0, 

where  A  £  RmXn  has  full  rank,  c  £  Rn,  and  b  £  Rm.  Feasible  points  x*  and  ( y*,s *) 
are  optimal  for  (LP)  and  (LD),  respectively,  if  and  only  if 

XjSj=0  for  j  =  l,2,...,n. 

We  assume  that  optimal  solutions  and  strictly  feasible  points  exist  for  both  problem 
(LP)  and  problem  (LD). 

Recently,  superlinear  convergence  results  have  been  obtained  for  LP  interior-point 
algorithms  by  several  authors,  e.g.  Kojimaetal.  [2],  and  Zhang  et  al.  [6], [7], [8].  These 
authors  have  shown  how  to  select  the  parameters  in  a  primal-dual  LP  interior-point 
algorithm  so  as  to  achieve  superlinear  convergence  under  the  assumption  that  the 
iterates  converge,  or  quadratic  convergence  under  the  assumption  of  nondegeneracy. 
Clearly,  it  is  desirable  for  an  algorithm  to  possess  the  global  property  of  polyno¬ 
mial  complexity  and  the  local  property  of  superlinear  convergence.  Interestingly,  the 
complexity  of  these  algorithms  is  at  best  0(nL)-iteration,  while  the  best  known  com¬ 
plexity  bound  is  0(\/nX)-iteration.  Here,  L  is  interpreted  as  the  precision  of  the  final 
solution.  Hence,  at  this  juncture  it  seems  reasonable  to  conjecture  that  the  price  one 
must  pay  for  superlinear  convergence  is  at  least  an  increase  in  the  complexity  bound 
by  a  factor  of  \/n.  The  purpose  of  this  technical  note  is  to  show  that  this  is  not 
the  case.  We  accomplish  this  objective  by  demonstrating  that  a  large  step  strategy 
first  proposed  by  Ye  [5]  can  be  adapted  to  the  Mizuno-Todd-Ye  predictor-corrector 
interior-point  algorithm  in  a  manner  that  maintains  its  0(y/nL )  iteration  complexity 
and  induces  superlinear  convergence  for  general  problems  and  quadratic  convergence 
for  nondegenerate  problems. 
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2.  The  Predictor-Corrector  Algorithm 

In  this  section,  we  briefly  describe  the  Mizuno-Todd-Ye  predictor-corrector  LP  algo¬ 
rithm  [4]  and  adapt  it  with  a  large  step  strategy.  We  employ  the  notation  X  =  diag(x), 
S  =  diag(s),  etc.  and  we  let  0  denote  the  collection  of  all  strictly  feasible  points  (x,  s ), 
i.e.,  (x,s)  >  0,  x  is  feasible  for  (LP),  and  s  is  feasible  for  (LD).  Consider  the  neigh¬ 
borhood 

Af(a)  —  {(x,s)  €  Q  :  || Xs/fi  —  e||  <  a}, 

where  j|.||  represents  the  I2  norm,  /x  =  xTs/n ,  e  is  the  vector  of  all  ones,  and  a  is  a 
constant  between  0  and  1. 

To  begin  with  choose  0  <  /3  <  1/2  (a  typical  choice  would  be  1/4).  All  search 
directions  dx,  ds,  and  dy  will  be  defined  as  solutions  of  the  following  system  of  linear 
equations  (Kojima  et  al.  [3]) 

Xds  +  Sdx  =  7/xe  —  Xs 

Adx  =  0  (1) 

ATdy  -f-  d3  =  0. 

A  typical  iteration  of  the  algorithm  proceeds  as  follows.  Given  (xk,sk)  €  A/"(/3), 
we  solve  the  system  (1)  with  (x,s)  =  (xk,sk)  and  7  =  0.  For  some  step  length  8  >  0 
let  x(0)  =  xk  +  8dx ,  s(8)  =  sk  +  8ds,  and  n(8)  —  x(0)T s{9)ln.  This  is  the  predictor 
step.  Our  specific  choice  for  6  will  be  stated  after  we  consider  the  following  lemma 
Lemma  1.  If  for  some  positive  9k  <  1  we  have 

|| X(8)s(8)/fx(8)  -  e\\  <  a  <  1  for  all  0  <  8  <  8k ,  (2) 

then  (x(8k),  s(8k))  €  Af(a). 

The  proof  of  Lemma  1  follows  directly  from  a  continuity  argument.  Lemma  1 
basically  says  that  the  feasibility  (positivity)  of  (x(#fc),  s(9k))  is  guaranteed  as  long 
as  (2)  is  satisfied.  Thus,  we  can  choose  the  largest  step  length  8k  <  1  such  that  (2)  is 
satisfied  for  a  =  2/3,  and  let 

x*  =  x(0*)  and  sk=s(8k). 
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This  choice  of  step  length  was  first  suggested  by  Ye  [5]. 

Now  we  solve  the  system  (1)  with  ( x,s )  =  ( xk,sk )  €  J\f(2 /?),  /i  =  (xk)T  sk /n, 
and  7  =  1.  Let  xk+1  —  xk  +  dx  and  sk+1  =  sk  +  ds.  It  has  been  proved  that 
(a;fc+1,sfc+1)  €  A/*(/?)  (Lemma  3  [4]).  This  is  the  corrector  step. 

Observe  that  the  algorithm  generates  a  sequence  of  points  satisfying 

(xk)Tsk  =  (l-9k)(xk)Tsk  (3.1) 


and 

(. xk+1)Tsk+1  =  ( xk)Tsk . 

Mizuno  et  al.  (Lemmas  1,  2,  and  4  [4])  showed  that  for  the  predictor  step 


(3.2) 


and 


ek^min{b{whu\r} 


\Dxda\\/ fik  <  y/2n/4. 


Thus,  this  inequality  together  with  (3)  implies  that  the  iteration  complexity  of  the 
algorithm  is  0{y/nL).  Note  that,  unfortunately,  the  algorithm  requires  that  the  linear 
system  (1)  be  solved  twice  at  each  iteration. 

The  above  lower  bound  is  not  sufficient  to  demonstrate  superlinear  convergence 
since  it  is  at  most  1/2.  Thus,  we  derive  another  bound  for  0k . 

Lemma  2.  If  9k  is  the  largest  6k  satisfying  the  conditions  of  Lemma  1  with  a  —  2/3, 
then 

9k  >  2  - . 

~  1/1+  A\\Dxds/ nk\\/  f3  +  1 

Proof.  In  the  predictor  step 

||X(0)s(0)  -  n(0)e ||  =  ||(1  -  9)(Xksk  -  /e)  +  92Dxds\\ 

<  ||(1  -9)(Xksk-fike)\\+  92\\Dxds\\ 

=  {l-9)\\(Xksk  -  nke)\\  +  02\\Dsd,\\ 

<  (1  -9)/3fik  +92\\Dxd3\\ 
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for  0  <  9  <  1.  Thus, 


(1  -  9)/3fxk  +  92\\Dxds\\  <  2/3(1  -  9)fik  =  2 fo(0)  (4) 

guarantees  the  satisfaction  of  inequality  (2)  for  a  =  2/3.  The  largest  6  satisfying  (4) 
is  the  positive  root  9+  of  the  quadratic  equation 

(l-9)/3fik +  02\\Dxds\\  =  2/3(1 -9)fik, 

which  is 

6+  =  . 2  - . 

\/l  +  M\Dxds/  (j,k\\//3  +  1 

Note  for  all  0  <  9  <  6+  strict  inequality  holds  in  (4),  and  also  in  (2)  with  a  =  2/3. 
This  concludes  the  proof  of  Lemma  2.  I 

3.  Superlinear  Convergence 

From  (3),  we  see  that  if  (1  —  9k)  — ►  0  then  the  duality  gap  ( xk)Tsk  converges  to  zero 
Q-superlinearly.  Moreover,  if  (1  -9k)  =  0((xk)T sk)  then  the  duality  gap  converges  to 
zero  Q-quadratically.  Hopefully,  we  can  accomplish  these  objectives  using  the  bound 
given  in  Lemma  2. 

At  the  kth.  predictor  step,  for  convenience,  let  6k  =  Dxdsj [ik .  If  9k  is  the  largest 
9k  satisfying  the  conditions  of  Lemma  1  with  a  =  2(3,  then 

i  -  <  i  —  2  — 

%/l  +  4||«‘||//3+l 

_  /l  +  4P*||//?-l 

V/U4ptP+l 

4||<tll//> 

(%/T+4pMiZ8  +  l)2 

<  P*ll AS-  (5) 

We  are  now  in  a  position  to  state  our  main  result. 
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Algorithm  (Large-step  predictor-corrector) 

By  the  large-step  predictor-corrector  algorithm  we  mean  the  Mizuno-Todd-Ye  al¬ 
gorithm  defined  in  Section  2  adapted  with  the  step  length  choice  as  the  largest  9k 
satisfying  the  conditions  of  Lemma  1  with  a  =  2/3. 

The  choice  of  9k  in  our  algorithm  will  involve  finding  the  roots  of  a  quartic  poly¬ 
nomial.  Following  the  proof  of  our  theorem  we  will  point  out  that  the  choice  for  9k 
need  not  be  this  involved  and  it  suffices  to  choose  8k  as  the  lower  bound  given  in 
Lemma  2. 

Theorem. 

(i)  The  Algorithm  has  iteration  complexity  0(\/nL). 

Let  {(xk ,  sk),  (xk,  ifc)}  be  generated  by  the  Algorithm.  Assume  that 

lim(:r*,.s*)  =  lim  (xk,sk)  =  ( x*,s *). 


Then 

(ii)  1  -  9k  ->  0. 

(iii)  ( xk)Tsk  — >  0  Q-superlinearly. 

(iv)  Xksk  — »  0,  component-wise,  Q-superlinearly. 

Assume  that  ( x*,s *)  is  nondegenerate.  Then 

(v)  1  ~9k  =  0((xk)Tsk). 

(vi)  ( xk)Tsk  — *■  0  Q-quadratically. 

Proof.  The  proof  of  (i),  i.e.,  the  0(-s/nX)-iteration  complexity  of  the  algorithm  is  as 
before. 

Since  the  iteration  sequence  {( xk,sk )}  converges  to  (z*,s*),  then  from  a  theorem 
of  G filer  and  Ye  [1]  (x*  ,s*)  must  be  a  strict  complementarity  solution  of  (LP)  and 
(LD).  This  is  guaranteed  because  all  ( xk,sk )  and  (xk,sk)  lie  in  Af(2 (3).  Without  loss 
of  generality,  assume  that 

xk>(  and  sk  — >  0  (6) 
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for  some  fixed  positive  number  £.  Now,  at  the  kth  predictor  step,  we  have  from  (1) 


xj(da)j  +  sj(dx)j  =  -tfs* 


or 


(ds)j  ( dx)j  _ 

U  I  i-  -*■  * 


k 

Xj 


From  our  convergence  assumption,  Lemma  2,  and  (6)  we  have 


Hence, 


Thus, 


so  that 


Xk:  -  Xkj 

,  =  lim  3—  ,J~  =  0. 

k—*oo  X ■  k— >oo  9*  X; 

3  3 


lim  {dx)j 


lim  ^  =  -1. 


k— "oo  S  • 
3 


lim  =  0, 


k-*  oo  XkSk 


lim  {}  =  lim  =  o, 

k—*oo  J  k—>-oo  fA* 


since  xksk  =  0(fik).  The  above  limit  holds  for  all  1  <  j  <  n.  Hence, 


lim  ||**||=0, 

k— *oo 


which  together  with  (5)  implies 


lim  (1  -  9k )  =  0. 

k—*oc 

This  establishes  (ii).  Conclusion  (iii)  follows  from  (3)  and  (ii),  and  in  turn  (iv)  follows 
from  (iii)  exactly  as  in  the  proof  of  Corollary  3.2  of  Zhang  et  al.  [7].  If  we  assume 
that  (a:*,s*)  is  nondegenerate,  then  from  Lemma  3.2  of  Zhang  et  al.  [7], 

(d,y*$=o((**)V) 


for  j  such  that  x*  >  0  and 


=  0((**)V) 
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for  j  such  that  x*-  =  0.  Thus,  for  all  j 


\{dx)j(ds)j\  =  0((x*)T5fc) 
x^s^ 

j  ] 

and 

|«)|  =  =  0((xl)V). 

fi 

This  estimate  together  with  (5)  gives  (v).  From  (3)  we  see  (vi)  follows  from  (v).  This 
proves  the  theorem.  I 

4.  Further  Remarks 

The  assumption  that  the  iteration  sequence  converges  is  used  in  all  other  superlinear 
convergence  results  (Kojima  et  al.  [2],  and  Zhang  et  al.  [6], [7], [8]).  Although  this 
property  is  “observed”  in  practice  for  LP  problems  whose  solution  set  is  bounded, 
it  is  a  challenging  open  question  as  to  what  conditions  imply  the  convergence  of  the 
iteration  sequence.  Under  the  assumption  of  nondegeneracy  the  LP  solution  pair  is 
unique.  This  coupled  with  the  facts  that  the  level  sets  of  the  duality  gap  function 
are  bounded  and  the  duality  gap  is  decreasing  monotonically  to  zero  implies  that 
the  iteration  sequence  converges.  This  is  implicitly  used  in  Theorem  7.1  of  Zhang 
and  Tapia  [6].  Hence  conclusions  (v)  and  (vi)  of  our  theorem  do  not  require  the 
convergence  assumption  made  in  the  theorem. 

From  Lemma  2  and  the  proof  of  the  Theorem,  it  follows  that  one  can  simply  choose 


yi+4pw  +  i 

and  not  search  for  the  largest  6k  satisfying  the  conditions  of  Lemma  1  with  a  =  2/3. 
The  global  and  local  behavior  of  the  algorithm  will  remain  the  same. 


8 


References 


[1]  O.  Giiler  and  Y.  Ye,  “Convergence  behavior  of  some  interior-point  algorithms,” 
Working  Paper  91-4,  The  College  of  Business  Administration,  The  University  of 
Iowa  (Iowa  City,  IA.),  1991. 

[2]  M.  Kojima,  Y.  Kurita  and  S.  Mizuno,  “Large  step  interior-point  algorithms  for 
linear  complementarity  problems,”  Research  Report  B-243,  Department  of  Infor¬ 
mation  Sciences,  Tokyo  Institute  of  Technology  (Tokyo,  Japan),  1991. 

[3]  M.  Kojima,  S.  Mizuno  and  A.  Yoshise,  “A  polynomial-time  algorithm  for  a  class 
of  linear  complementarity  problems,”  Mathematical  Programming  44  (1989)  1-26. 

[4]  S.  Mizuno,  M.  J.  Todd,  and  Y.  Ye,  “On  adaptive-step  primal-dual  interior-point 
algorithms  for  linear  programming,”  Technical  Report  No.  944,  School  of  ORIE, 
Cornell  University  (Ithaca,  NY.),  1990,  to  appear  in  Mathematics  of  Operations 
Research. 

[5]  Y.  Ye,  “Further  developments  on  interior  algorithms  for  convex  quadratic  program¬ 
ming,”  unpublished  manuscript,  Department  of  Engineering-Economic  Systems, 
Stanford  University  (Stanford,  CA.),  1987. 

[6]  Y.  Zhang  and  R.  A.  Tapia,  “A  quadratically  convergent  polynomial  primal-dual 
interior-point  algorithm  for  linear  programming,”  TR90-40,  Department  of  Math¬ 
ematical  Sciences,  Rice  University  (Houston,  TX.),  1990. 

[7]  Y.  Zhang,  R.  A.  Tapia  and  J.  E.  Dennis,  “On  the  superlinear  and  quadratic  con¬ 
vergence  of  primal-dual  interior-point  linear  programming  algorithms,”  TR90-6, 
Department  of  Mathematical  Sciences,  Rice  University  (Houston,  TX.),  1990. 

[8]  Y.  Zhang,  R.  A.  Tapia  and  F.  Potra,  “On  the  superlinear  convergence  of  interior- 
point  algorithms  for  a  general  class  of  problems,”  TR90-7,  Department  of  Mathe¬ 
matical  Sciences,  Rice  University  (Houston,  TX.)  1990. 


9 


